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I.  INTRODUCTION 


This  chapter  will  help  set  the  stage  for  the  purpose  of  this  research  and 
subsequently  serve  as  a  guide  for  what  to  expect  in  the  succeeding  chapters.  As 
the  Army  contracting  workforce  has  increased  over  the  last  decade,  during  a  time 
of  war,  there  has  been  an  abundance  of  contact  mismanagement,  fraud,  and 
unethical  behavior  from  the  acquisition  contracting  workforce,  which  has  cost 
billions  of  dollars  in  lost  capital.  Current  remedies  include,  rightsizing  the 
contracting  workforce,  additional  training,  incentivizing  performance,  and 
reaffirming  ethical  standards,  but  these  initiatives  are  incremental  solutions  and 
only  focus  on  individual  competence.  What  is  called  for  is  a  review  of  contract 
management  process  capability.  This  research  will  show  that  by  assessing 
contract  management  process  capability  using  the  Contract  Management 
Maturity  Model  (CMMM),  an  organization  can  view  their  contract  management 
processes,  identify  shortcomings,  and  take  corrective  actions  to  improve  the 
organizations  contract  management  process  capability.  We  will  look  at  Aberdeen 
Proving  Ground-Army  Contracting  Command  (APG-ACC)  contract  management 
processes,  using  the  CMMM. 

A.  BACKGROUND 

The  Army’s  mission  has  changed  significantly  since  the  drawdown  of 
forces  at  the  end  of  the  Cold  War.  The  Army  saw  a  large  exodus  of  its  military 
personnel  in  the  1990s  due  to  a  drawdown  in  forces;  however  there  was  an 
increase  in  contracted  support  to  augment  the  force.  In  addition  to  an  increase  in 
contracted  support  the  Army  also  increased  its  contract  spending  (Figure  1). 
After  September  11,  2001,  the  environment  transformed  yet  again  from  a  post 
Cold  War  government  workforce,  to  an  expeditionary  workforce  focused  on  the 
deployed  Warfighter,  DOD  civilian,  and  contractor  support.  It  was  then  realized 
that  the  government  contracting  force  was  ill  prepared  to  provide  quality 
contracting  support  to  deployed  forces.  According  to  the  Gansler  Report  “The 
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Institutional  Army  has  not  adjusted  to  the  challenges  of  providing  timely,  efficient, 
and  effective  contracting  support  to  the  force  in  Operation  Iraqi  Freedom,  more 
than  half  of  which  is  contractor  personnel”  (Gansler  Report,  2007). 

Although  the  Army’s  operational  environment  and  mission  have  evolved, 
what  did  not  evolve  were  the  competencies  of  the  government  contracting 
workforce.  The  Army’s  undeveloped  contracting  workforce  found  itself 
responsible  for  procuring  services  and  supplies  in  austere  environments  and 
managing  enormous  budgets  while  doing  so.  As  noted  by  Congress,  DOD 
contracting  has  been  plagued  with  issues  of  accountability,  poor  oversight  of 
contractor  performance,  fraud,  theft,  and  ethical  violations,  (United  States 
Congress  Senate  Committee  on  Appropriations,  2008). 

In  2007,  the  Secretary  of  the  Army  established  a  commission  to  look  at 
Army  Acquisition  and  Program  Management  support  to  expeditionary  operations, 
so  that  shortcomings  could  be  addressed  to  provide  better  contracting  support. 
Corrective  actions  that  were  recommended  were  the  following:  1.  Increase  the 
military  workforce  and  include  General  Officer  (GO)  billets;  2.  Restructure/right 
size  the  organization;  3.  Provide  training  and  tools  for  success;  4.  Obtain 
legislative,  regulatory,  and  policy  assistance  (Commission  on  Army  Acquisition 
and  Program  Management  in  Expeditionary  Operations  (2007).  Seven  years 
after  the  release  of  the  Gansler  Report  and  implementation  of  recommend 
actions;  contract  mismanagement  is  an  issue  that  continues  to  plague  the  military 
services  (GAO,  2013). 
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Figure  1.  Army  Contract  Spending,  1990-2010  (from  DD350  and  FPDS; 

CSIS  analysis  (2011) 


B.  PURPOSE 

The  purpose  of  this  research  is  to  assess  the  contract  management 
processes  at  Aberdeen  Proving  Ground-Army  Contracting  Command  (APG- 
ACC)  by  surveying  its  subordinate  contracting  centers  using  the  Contract 
Management  Maturity  Model  (CMMM).  Through  in-depth  analysis,  the  research 
will  assess  APG-ACC  contract  management  process  capability  by  analyzing  the 
integration  of  best  practices  throughout  the  contracting  process.  We  will  also 
cross-reference  the  prescribed  government  compliance  metrics  with  the  CMMM 
assessment  results.  We  will  survey  each  subordinate  contracting  center,  analyze 
the  data  received,  and  then  provide  an  overall  rating  to  the  APG-ACC  contracting 
management  command.  The  results  from  the  research  will  help  APG-ACC 
identify  strengths  and  weakness  in  its  contract  management  processes. 
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C.  RESEARCH  QUESTIONS 

To  better  understand  the  contract  management  process  capability  at  APG- 
ACC  we  will  answer  the  following  questions: 

1 .  What  is  the  contract  management  process  maturity  level  for  APG- 
ACC  in  each  of  the  six  process  areas  of  the  CMMM? 

2.  What  process  improvement  opportunities  are  available  for  APG- 
ACC,  based  on  the  CMMM  assessment  at  APG? 

3.  What  metrics  does  APG  currently  use  to  assess  contract 
management  performance? 

4.  How  do  compliance  reports  of  APG-ACC  compare  to  the  CMMM 

results? 

D.  ORGANIZATION  OF  REPORT 

This  report  is  comprised  of  the  following  five  chapters:  Chapter  I  describes 
the  purpose  of  the  research;  introduces  the  research  questions;  and  discusses 
the  significance  and  implications  of  the  study.  Lastly,  in  Chapter  I  we  will  discuss 
limitations  to  this  research  project  and  the  benefits  gained  from  conducting  the 
study. 

Chapter  II  is  a  review  of  the  literature.  In  the  literature  review  we  discuss 
why  organizations  should  measure  performance;  compliance  to  public  policy  and 
agency  regulations;  how  should  we  measure;  and  what  should  we  measure. 
Furthermore,  we  will  discuss  the  Contract  Management  Maturity  Model,  its  key 
process  areas  and  maturity  level  descriptions. 

In  Chapter  III,  we  provide  an  overview  of  APG-ACC  organizational 
construct,  its  mission,  and  the  services  the  organization  provides  its  customers. 
We  also  discuss  metrics  currently  being  used  by  the  organization. 

Chapter  IV  describes  the  findings  from  the  CMMM  assessment.  In 
Chapter  IV,  we  assign  a  maturity  level  to  the  organization’s  contract  management 
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process  areas.  Additionally,  we  will  cross-reference  APG-ACC  CMMM  results 
with  DOD  Inspector  General  (IG)  and  Government  Accountability  Office  reports 
to  see  if  there  are  consistencies  between  the  reports  and  the  assessment. 

Chapter  V  will  summarize  the  report  and  provide  recommendations  for 
areas  of  further  research. 

E.  METHODOLOGY 

We  will  conduct  a  literature  review  on  why  organizations  should  measure 
performance,  what  should  be  measured,  and  how  performance  should  be 
measured.  We  will  also  cover  the  CMMM  and  its  previous  applications  with  other 
Army  organizations  in  our  literature  review. 

We  deployed  a  62-question  survey  to  assess  the  contract  management 
process  maturity  at  APG-ACC.  We  analyzed  the  data  from  the  survey  and 
assigned  a  contract  management  maturity  level  to  each  of  the  six  phases  of  the 
contract  management  process. 

The  target  populations  for  our  research  were  APG-ACC  1102  series 
workers  and  military  equivalents  who  are  DAWIA  level  II  and  level  III  certified  in 
contracting.  We  selected  this  population  because  level  II  and  level  III  certified 
individuals  were  more  familiar  with  the  contracting  processes  at  APG-ACC  due  to 
longevity  in  the  organization  and  the  contracting  community.  Level  I  certified 
personnel  are  normally  the  entry-level  workers  of  a  contracting  organization,  and 
lack  the  experience  and  knowledge  of  that  the  level  II  and  III  contracting 
personnel  possess. 

To  assess  APG’s  current  contract  management  performance  we  will 
gather  the  metrics  they  currently  use.  We  will  also  explore  any  compliance 
reports  for  additional  consistencies  with  the  CMMM  assessment  results. 

F.  BENEFITS  OF  RESEARCH 

APG-ACC  will  be  the  primary  beneficiary  of  this  research.  This  report  will 

provide  APG-  ACC  leadership  an  assessment  of  their  organization  using  an 
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assessment  tool  outside  of  metrics  they  currently  use  collect.  APG-ACC  can 
compare  our  assessment  from  the  CMMM  to  their  organizational  performance 
metrics  to  see  if  there  are  consistencies  between  the  two  methods  used  to 
assess  the  organization.  Also,  using  this  report  APG-ACC  can  structure  their 
contract  management  initiatives  to  improve  contract  management  process  areas 
having  lower  levels  of  process  maturity,  and  implement  practices  to  sustain  areas 
that  have  been  identified  as  having  higher  levels  of  process  maturity. 

As  a  final  point,  the  results  from  this  research  can  be  used  as  a  baseline 
for  additional  research  being  conducted  within  the  APG-ACC.  The  CMMM  is  the 
only  contract  management  process  model  used  by  DOD  and  industry.  If  the 
organization  should  use  another  assessment  approach  to  gauge  the  overall 
contract  management  process  capability  they  can  make  comparisons  between 
the  two  assessment  findings  as  a  means  for  identifying  consistencies. 
Additionally,  this  research  can  be  used  by  other  Army  contracting  command 
organizations  and  other  DOD  contracting  agencies  as  an  example  of  assessing 
contract  management  process  capability. 

G.  LIMITATIONS  TO  RESEARCH 

There  are  several  limitations  that  we  identified  during  the  planning  and 
implementation  of  this  research.  The  first  limitation  identified  was  the  CMMM 
survey  was  sent  to  the  entire  workforce  within  the  divisions  identified  as 
performing  contracting  duties.  We  asked  that  only  level  II  and  level  III,  DAWIA 
certified  contracting  officers  and  specialists  take  the  survey.  There  may  have 
been  level  I  certified  individuals  who  took  the  survey  and  their  responses  will  be 
included  in  the  results.  The  next  limitation  identified  was  our  use  of  only  the 
CMMM  to  identify  best  practices  and  shortcomings.  There  are  other  assessment 
tools,  such  as  the  DOD  IG  compliance  inspection  guides  and  organizational 
performance  metrics  that  could  have  been  used  to  measure  organizational 
contract  compliance,  goals,  and  management.  We  also  add  the  assumption  that 
survey  respondents  are  truthful  and  honest  in  answering  survey  questions  as  a 
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limitation  to  this  research.  A  final  limitation  is  that  although  we  encouraged  full 
participation  in  the  survey  we  received  a  response  rate  of  17.04%. 


H.  SUMMARY 

In  this  chapter,  we  introduced  the  research  topic;  provided  background 
information;  the  purpose  of  the  research;  the  questions  we  set  out  to  answer;  the 
organization  of  the  report;  methodology;  benefits  of  the  research;  and  limitations 
of  the  research.  The  next  chapter  is  a  review  of  the  literature.  In  the  literature 
review  we  discuss  why  organizations  should  measure  performance;  compliance 
to  public  policy  and  agency  regulations;  how  organizations  should  measure 
performance;  and  what  should  be  measured.  Furthermore,  we  will  discuss  the 
CMMM,  its  key  process  areas,  and  maturity  level  descriptions. 
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II.  LITERATURE  REVIEW 


A.  INTRODUCTION 

Organizations  should  measure  performance  to  fully  subscribe  to 
establishing  efficiency  and  effectiveness  within  their  organization.  This  chapter 
looks  to  provide  a  literature  review  of  public  purchasing  performance  evaluation 
and  its  significance.  We  further  reviewed  various  evaluation  frameworks  available 
to  include  using  performance  metrics,  evaluating  compliance  and  best  practice 
models. 

B.  PUBLIC  PROCUREMENT  PERFORMANCE  MEASUREMENT  AND 

EVALUATION 

1.  Why  Should  We  Measure? 

Procurement  equates  to  nearly  one-third  of  all  government  outlays, 
making  it  particularly  sensitive  to  accountability  and  transparency  within  public 
administration  (Schapper  et  al.,  2006).  “Value  for  money  is  the  core  principle 
governing  public  procurement  and  is  supported  by  the  underpinning  principles  of 
efficiency  and  effectiveness,  competition,  accountability,  transparency,  ethics  and 
industry  development”  (Raymond,  2008,  p.782).  In  turn,  public  entities  should 
strive  to  achieve  the  “value  for  money”  for  the  public  through  use  of  elements  to 
include  customer  satisfaction  and  the  public  interest  (Raymond,  2008).  By  law, 
federal  agencies  are  required  to  conduct  performance  measures  to  show  the 
strategy  for  achieving  value  for  money  and  providing  the  highest  quality  services 
and  products  to  the  customer  (Cavanagh  et  al.,  1999). 

Weele  (2010)  conducted  a  survey  of  purchasing  managers  and  their 
views  regarding  what  benefits  an  organization  could  realize  in  evaluating 
purchasing  performance.  The  responses  provided  proposed  better  decision 
making,  improved  communication  between  departments,  accountability  and 
professional  development  as  advantages  to  evaluating  purchasing  performance 
(Weele,  2010).  Weele  further  concludes  that  there  are  two  primary  reasons  to 
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assess  performance:  to  rate  the  individual  buyer  for  individual  performance 
assessment  and  to  allow  the  individual  buyer  to  assess  how  their  efforts  are 
contributing  to  the  organizations  goals  and  objectives  (2010). 

Purchasing  can  have  a  pivotal  role  in  overall  organizational  function 
depending  on  how  it  is  viewed  by  management.  The  organizational  leadership’s 
view  of  how  purchasing  fits  into  the  agency  structure  influences  the  way 
performance  measurements  are  determined.  Weele  (2010)  discusses  that  there 
are  four  functional  perceptions  that  the  leadership  can  hold  regarding  purchasing: 
operational  or  administrative,  commercial,  integrated  logistics  management  and 
as  a  strategic  business  function.  These  perceptions  further  translate  into  where 
purchasing  ranks  within  the  organization. 

The  lowest  tier  view  is  derived  from  the  operational  administrative  function 
and  results  in  measurements  reflecting  the  clerical  aspects  of  purchasing.  With 
the  commercial  function,  Weele  (2010)  suggests  that  purchasing  is  a  direct  report 
to  management  and  expands  metrics  to  an  organization’s  goals  as  they  relate  to 
savings,  return  on  investment  and  price  reductions.  The  next  perspective  weaves 
purchasing  into  other  material  functions  within  the  organization  resulting  in 
measurements  that  involve  delivery  reliability  and  lead-time  reduction.  The  final 
perspective  includes  purchasing  in  top  tier  management  and  propels  the 
measurements  to  more  proactive  elements  such  as  early  supplier  involvement 
and  “should-cost”  analysis. 

2.  What  Should  We  Measure? 

Cohen  and  Emieke  propose  four  categories  of  performance  metrics  and 
that  most  successful  performance  management  systems  use  each  of  them: 
inputs,  processes,  outputs  and  outcomes  (2008).  Inputs  are  the  easiest  to 
measure  because  of  the  ability  to  collect  and  identify  these  metrics.  They  are 
significant  to  an  organization’s  ability  to  assess  demand  of  resources,  reflect 
organizational  priorities  and  customer  preferences.  Contracting  organizations  can 
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collect  information  such  as  certification  level,  years  of  experience,  and  dollars 
appropriated  or  budgeted  for  requirements  as  valuable  inputs. 

Process  measurement  addresses  the  way  the  work  is  being  conducted 
and  helps  an  organization  in  determining  quality  of  production  relative  to 
efficiency  (Cohen  &  Emieke  2008).  It  is  this  type  of  measurement  that  connects 
organizational  improvement  and  learning  by  taking  into  account  activities  such  as 
production  rates,  error  rates,  and  number  steps  or  tasks  required  to  complete 
work  (Cohen  &  Emieke  2008).  Federal  contracting  agencies  use  Procurement 
Administrative  Lead  Time  also  known  as  PALT  to  measure  the  duration  a 
requirement  takes  to  be  procured. 

The  output  measurement  provides  information  on  the  quantity  of  work 
being  performed  as  it  relates  to  the  inputs  and  resources  available.  Output  data, 
when  utilized  with  multiple  indicators,  informs  an  organization  on  several  facets 
of  performance  and  a  more  detailed  analysis  of  outputs  (Cohen  &  Emieke  2008). 
Indicators  such  as  dollars  obligated,  contracts  closed  out  or  completed  could  be 
used  by  a  contracting  organization  to  capture  outputs. 

The  final  performance  measurements  are  outcomes,  also  identified  as 
impacts.  These  performance  indicators  seek  to  identify  effectiveness  by  tying 
inputs  and  outputs  to  the  organization’s  goals.  Outcomes  are  the  hardest  factors 
to  measure,  as  it  takes  longer  to  realize  the  impact  of  an  action  and  require  the 
customer’s  perspective,  however  provides  the  most  valuable  information  to  the 
organization  (Cohen  &  Emieke  2008).  Purchasing  organizations  can  tie  in  socio¬ 
economic  and  small  business  goals  to  measureable  outcomes  linked  to  their 
higher  mission  to  support  customers  and  public  interest. 

3.  How  Should  We  Measure? 

Before  measuring  purchasing  performance,  Weele  (2010)  advises  that 
performance  must  be  classified  using  two  elements:  effectiveness  and  efficiency. 
Purchasing  effectiveness  relates  to  “the  extent  which,  by  choosing  a  certain 
course  of  action,  a  previously  established  goal  or  standard  is  being  met,”  (Weele, 
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2010,  p.  303).  This  outcome  answers  whether  the  purchasing  activities  are 
meeting  the  set  end-state.  Purchasing  efficiency  is  “the  relationship  between 
planned  and  actual  sacrifices  made  in  order  to  realize  a  goal  previously  agreed 
upon,”  (Weele,  2010,  p.  305).  Efficiency  is  the  relationship  between  actual  costs 
and  planned  costs.  Together,  efficiency  and  effectiveness  equal  performance 
(Weele,  2010). 

Weele  (2010)  further  categorizes  performance  measurement  into  four  key 
areas:  purchasing  material  costs/  prices,  product  quality,  purchase  logistics  and 
purchasing  functions.  The  first  three  are  subsets  of  purchasing  effectiveness  and 
the  last  a  subset  of  purchasing  efficiency  that  are  further  narrowed  down  to 
actions  that  should  be  evaluated. 

Regulatory  compliance  is  a  method  for  evaluating  performance  and  is 
noted  as  a  traditional  framework  in  public  procurement;  however,  management 
can  only  expect  to  maintain  status  quo  with  regard  to  performance  using  this 
method  alone  (Schapper  et  al. ,  2006).  Policy  compliance  can  examine 
“processes  and  outcomes  of  procurement  in  relation  to  other  policy  objectives 
and  expectations  of  the  public  sector”  (Schapper  et  al,  2006,  p.5).  DOD  assesses 
compliance  through  agencies  such  as  the  Inspector  General  and  Congress  who 
maintains  oversight  of  policy  compliance  through  the  Government  Accountability 
Office. 

Benchmarking  is  a  form  of  performance  evaluation  that  can  be  described 
as  a  method  that  attempts  to  “visualize  best  practices  through  normalizing 
comparison  and  by  urging  public  entities  to  ask  themselves  what  can  they  do  to 
promote  ‘best  practices’”  (Raymond,  2008).  Organizations  can  determine  and 
compare  their  best  practices,  identify  and  improve  gaps  and  current  performance 
levels  (Raymond,  2008). 

There  are  several  studies  that  support  the  method  of  benchmarking  to 
determine  purchasing  best  practices  within  an  organization.  Brandmeier  and 
Rupp  cited  research  from  three  separate  studies  where  the  impact  of 
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benchmarking  revealed  increase  public  procurement  performance.  Brandmeier 
and  Rupp’s  (2010)  research  expanded  the  area  by  focusing  on  specific  practices 
that  increased  performance  by  evaluating  companies  already  known  for 
achieving  high  levels  of  performance.  They  concluded  certain  benchmarking 
success  factors  were  particularly  salient  to  the  procurement  process  that 
included:  cross  functional  teams,  high  placement  in  the  organizational  structure, 
cooperation  with  other  functions,  supplier  integration  into  training  and 
development  of  procurement  workforce  and  reoccurring  evaluation.  In  a  case 
study  analysis  of  how  benchmarking  could  be  conducted  within  the  procurement 
process,  Raymond  found  that  organizations  should  include  measures  to  address 
accountability,  transparency,  value  for  money,  workforce  professionalism  and 
ethics  in  order  to  enact  improvement  to  public  procurement  systems  (2008). 

The  Balance  Scorecard  approach  is  presented  by  Niven  as  “a  carefully 
selected  set  of  quantifiable  measures  derived  from  the  organizations  strategy” 
(2003,  p.  14)  that  resulted  in  the  public  sector  desire  to  improve  measures  of 
organizational  performance.  The  framework  analyzes  four  areas  of  emphasis:  the 
customer,  internal  processes,  learning  and  growth  and  financial  (Niven,  2003). 
Both  public  and  private  entities  recognize  the  need  to  connect  performance 
measurement  into  management  of  the  organization  in  order  to  obtain  the 
competitive  advantage  in  their  respective  industries.  In  1993,  the  Procurement 
Executives  Association  (PEA)  developed  a  Procurement  Measurement 
Assessment  Team  that  utilized  the  balance  score  card  methodology  to  assess 
the  federal  acquisition  system,  determine  best  practices  in  measuring 
performance  and  identify  strategies  and  recommendations  to  improve  the  state  of 
acquisition  agencies  (Cavanagh  et  al. ,  1999).  The  team  determined  that  federal 
performance  management  systems  needed  the  following  qualities  to  achieve  the 
measurement  into  management  connection:  1.  Agency  vision  translated  into 
measureable  outcomes  that  have  been  socialized  with  the  rest  of  the  agency, 
stakeholders  and  customers;  2.  An  appropriate  assessment  tool  that  incorporates 
management  and  improvements;  3.  Shift  from  compliance  and  audit-based 
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oversight  to  “forward-looking  strategic  partnerships”  (Cavanagh  et  al.,  1999, 
p.14);  4.  Incorporate  multiple  measurements  from  customer  service  to  quality  and 
cost;  5.  Adopt  a  consistent  approach  to  performance  management  (Cavanagh  et 
al.,  1999). 

The  contract  score  card  developed  by  Cullen  looks  further  past  the 
balance  scorecard  approach  by  assessing  an  organization’s  contracted  functions 
(2009).  She  argues  that  the  balance  scorecard  only  accurately  informs  an 
organization  of  its  internal  functions,  where  the  contract  scorecard  measures  the 
external  arrangements  of  an  agency  (Cullen,  2009). 

Maturity  Models  are  methods  of  process  measurement  that  capture  an 
organization’s  development  over  time  and  assess  that  development  through 
four  factors:  “The  development  of  a  single  entity  is  simplified  and  described  with 
a  limited  number  of  maturity  levels  (usually  four  to  six).  The  maturity  levels  are 
characterized  by  certain  requirements  which  the  entity  has  to  achieve  on 
that  level.  Levels  are  sequentially  ordered,  from  an  initial  level  up  to  an  ending 
level  (the  latter  is  the  level  of  perfection).  During  development  the  entity  is 
progressing  forwards  from  one  level  to  the  next  one.  No  levels  can  be  left  out” 
(Weerdmeester  et  al.,  2003,  p.  5). 

Various  sectors  of  industry  have  developed  maturity  models  to  measure 
organizational  progression.  These  models  include  Project  Management  Maturity, 
Earned  Value  Management  Maturity,  Software  Management  Maturity,  Capability 
Maturity,  Knowledge  Management  Maturity,  and  Contract  Management  Maturity, 
which  is  the  basis  for  our  research  and  will  be  discussed  next. 

C.  CONTRACT  MANAGEMENT  MATURITY  MODEL 

The  CMMM  was  developed  by  Rendon  in  2003,  for  DOD  to  specifically 
assess  best  practices  as  they  relate  to  levels  of  organizational  contract 
management  process  maturity  across  the  six  phases  of  the  contract 
management  process.  Contract  management  maturity  is  the  relationship 
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between  the  process  capabilities  of  an  organization  and  its  ability  to  produce 
successful  results  between  buyers  and  sellers  (Rendon,  2008). 

1 .  Key  Process  Areas 

Garrett  and  Rendon  proposed  that  “in  order  to  award  and  successfully 
manage  effective  contracts,  organizations  must  have  disciplined,  capable,  and 
mature  contract  management  processes  in  place”  (2005,  p.  48).  To  further 
analyze  maturity,  the  CMMM  examines  the  phases  of  contract  management  as 
key  process  areas  to  evaluate.  Appendix  A  references  the  contract  phases  with 
the  corresponding  activities  and  applicable  Federal  Acquisition  Regulation  Parts. 
Past  studies  using  the  CMMM  indicate  that  low  maturity  in  any  contract 
management  process  area  contributes  to  the  detriment  of  effectiveness  and 
efficiency  within  an  organization.  The  CMMM  includes  six  phases  of  the  contract 
management  process  which  are  discussed  next. 

Procurement  Planning  is  the  internal  identification  of  which  services  or 
products  to  procure,  how  to  procure  them,  when  to  procure  them  and  at  what 
cost.  This  contracting  phase  includes  the  following  activities  (Rendon,  2011): 

(1)  Conducting  requirements  analysis 

(2)  Determining  required  sources  of  supply  and  services 

(3)  Conducting  Acquisition  Planning 

(4)  Conducting  Market  research 

(5)  Determining  competition  environment 

Solicitation  Planning  is  the  process  of  identifying  project  or  program 
requirements,  documentation  and  potential  vendors  that  can  provide  the 
requirement.  In  this  phase  the  following  key  activities  are  conducted  (Rendon, 
2011): 

1.  Documenting  competition  environment 

2.  Determining  the  procurement  method 
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3.  Determining  evaluation  strategy 

4.  Developing  solicitation  documents 

5.  Determining  contract  type/  incentive 

6.  Determining  terms  and  conditions 

Solicitation  is  the  process  of  obtaining  bids  or  proposals  from  potential 
suppliers  that  will  meet  internal  organizational  needs.  The  phase  contains  the 
following  activities  (Rendon,  2011): 

1 .  Advertising  the  procurement 

2.  Conducting  conferences  (pre-solicitation,  pre-proposal) 

3.  Amending  solicitation  documents  as  required 

Source  Selection  is  evaluating  the  potential  bids  against  established 
criteria  to  select  a  vendor.  The  key  activities  for  this  phase  are  as  follows 
(Rendon,  201 1 ): 

1 .  Evaluating  proposals 

2.  Applying  evaluation  criteria 

3.  Negotiating  contract  terms  and  conditions 

4.  Determining  contactor  responsibility  standards 

5.  Selecting  contractor 

6.  Managing  protests,  disputes  and  appeals 

Contract  Administration  is  the  process  of  ensuring  the  contractual 
obligation  of  each  party’s  performance.  The  following  activities  occur  if 
applicable,  during  this  phase  (Rendon,  2011): 

1.  Conducting  conferences  (post-award,  pre-performance) 

2.  Managing  contract  change  process 

3.  Monitoring  contractor’s  management  of  subcontracting 
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4.  Managing  government  furnished  property 

5.  Monitoring  and  measure  contractor’s  performance 

6.  Managing  transportation  issues 

7.  Managing  value  engineering  issues 

8.  Managing  contractor  payment  process 

9.  Managing  contractor  payment  process 

10.  Managing  protests,  disputes  and  appeals 

1 1 .  Complying  with  terms  and  conditions 

Finally,  Contract  Closeout  is  the  settlement  of  the  contract  and  resolving  of 
any  outstanding  administrative  actions.  The  activities  that  occur  in  this  phase  are 
as  follows  (Rendon,  201 1 ): 

1 .  Verifying  contract  completion 

2.  Verifying  contractor  compliance 

3.  Ensuring  contract  completion  documentation 

4.  Making  final  payment 

5.  Documenting  lessons  learned/  best  practices 

6.  Processing  contract  terminations 

7.  Disposing  of  buyer-furnished  property  and  equipment 

8.  Processing  contract  closeout  procedures 

In  addition  to  the  six  phases  of  contract  management,  the  CMMM  also 
identifies  five  levels  of  process  maturity  and  these  levels  are  discussed  next. 

2.  Levels  of  Maturity 

There  are  five  process  maturity  levels:  Ad  Hoc,  Basic,  Structured, 
Integrated  and  Optimized. 
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Ad  Hoc:  Represents  the  initial  level  of  contract  management  processes 
that  are  informally  practiced  and  applied.  The  leadership  understands  the 
benefits  of  the  contract  management  process  but,  organization-wide  practices 
are  not  established.  There  may  be  some  processes  present  in  the  organization, 
however  no  consistency  to  when  they  are  applied.  Lastly,  there  is  a  lack  of 
accountability  held  to  leadership  and  contract  management  personnel  for  failure 
to  comply  with  basic  procurement  management  standards  (Rendon,  2011). 

Basic:  The  level  at  which  rudimentary  contract  processes  and  standards 
are  established  but  only  required  on  selected  contracts  that  meet  certain  criteria. 
Some  documentation  has  been  set  for  processes  however,  these  developments 
are  not  considered  institutionalized.  Finally,  organizational  policy  and  standards 
established  are  not  required  to  be  consistently  applied  for  any  contracts  that  have 
not  met  certain  criteria  (Rendon,  2011). 

Structured:  The  maturity  level  where  contract  management  processes  are 
institutionalized  and  formal  documentation  is  established  for  processes  and 
standards.  Documentation  is  present  for  formal  standards  and  possibly 
automated.  There  is  flexibility  in  application  of  processes  and  documents  to  fit 
various  types  of  contract  situations  since  contract  process  use  is  mandated.  With 
this  maturity  level,  the  organizational  leadership  is  well  versed  in  providing 
guidance  and  direction  for  strategy  development  and  issues  related  to  contract 
management  (Rendon,  2011). 

Integrated:  The  maturity  level  consists  of  fully  integrated  contract 
management  processes  that  are  tied  into  other  core  capabilities.  The  customer  is 
also  integrated  into  the  purchasing  team  and  leadership  utilizes  performance 
metrics  to  evaluate  the  contract  management  process  and  make  decisions 
(Rendon,  201 1 ). 

Optimized:  The  highest  level  of  maturity  “that  reflects  an  organization 
whose  management  systemically  uses  performance  metrics  to  measure  the 
quality  and  evaluate  efficiency  and  effectiveness  of  the  contract  management 
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process”  (Rendon,  2011,  p.15).  Best  practices  and  lessons  learned  are  fully 
integrated  into  organizational  operations  (Rendon,  2011).  The  results  of  the 
CMMM  can  be  further  analyzed  to  help  an  organization  focus  its  continuous 
improvement  efforts  by  identifying  key  process  enablers  which  will  be  discussed 
next. 


3.  Process  Enablers 

The  process  enablers  are  classified  into  five  areas:  Process  Strength, 
Process  Results,  Management  Support,  Process  Integration,  and  Process 
Measurement  (Rendon,  2011). 

Process  Strength  is  measured  by  the  first  three  survey  items  in  each  key 
process  area.  Process  Strength  assesses  how  established  contract  management 
processes  are  and  the  level  at  which  they  are  standardized  and  documented. 

Process  Results  are  measured  by  the  fourth  survey  item  in  each  key 
process  area  as  well  as  the  sixth  and  seventh  items  in  the  area  of  source 
selection.  Process  Results  assess  the  success  of  outcomes  in  each  area,  such 
as  structuring  solicitations  to  facilitate  complete  and  accurate  proposals,  using 
appropriate  evaluation  criteria,  and  evaluating  past  performance  and  technical 
capability  in  contractor  proposal  evaluation. 

Management  Support  is  measure  by  the  fifth  survey  item  in  each  key 
process  area.  Management  Support  assesses  concerns  such  as  senior- 
management  involvement  in  providing  input  and  approval  of  key  planning 
decisions  and  documents. 

Process  Integration  is  measured  by  the  sixth,  seventh,  and  eighth  survey 
items  in  the  area  of  Procurement  Planning,  Solicitation  Planning,  and  Solicitation. 
Process  Integration  is  measure  by  the  eighth  and  ninth  survey  items  in  the  area 
of  Source  Selection,  the  sixth  through  the  ninth  survey  items  in  the  area  of 
Contract  Administration,  and  the  seventh  survey  item  in  the  area  of  Contract 
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Closeout.  Process  Integration  assesses  how  processes  are  integrated  across 
each  of  the  key  process  areas. 

Process  Measurement  is  measured  by  the  final  two  survey  items  in  each 
key  process  area  as  well  as  the  eighth  survey  item  in  the  area  of  Contract 
Closeout.  Process  Measurement  assesses  concerns  such  as  the  efficiency  and 
effectiveness  of  metrics  in  process  evaluation  and  improvement  (Chang,  Levine 
&  Philaphandeth,  2012). 

D.  PREVIOUS  APPLICATIONS  OF  CMMM  WITH  ARMY  ORGANIZATIONS 

The  CMMM  has  been  applied  to  other  contracting  organizations  within  the 
Army.  In  September  of  2009,  Kevin  Puma  and  Beth  Scherr  conducted  a  joint- 
applied  project  Assessing  Combat  Management  Maturity:  U.S.  Army  Joint 
Munitions  and  Lethality  Contracting  Center,  Army  Contracting  Command, 
Pictanny  Aresenal.  Their  study  of  six  contracting  organizations  found  that  all  of 
the  sub-centers  operated  at  the  Integrated  or  Structured  maturity  level  for  all 
phases  of  contract  management,  less  Contract  Closeout.  The  Contract  Closeout 
phase  was  recorded  at  the  Basic  level  and  the  researchers  recommended  the 
organizational  leadership  focus  on  key  process  areas  and  continually  monitor  the 
reform  efforts  (Puma  &  Scherr,  2009). 

Dina  Jeffers  (2009)  conducted  research  on  Contract  Specialist  Turnover 
Rate  and  Contract  Management  Maturity  in  the  National  Capital  Region 
Contracting  Center:  An  Analysis.  Jeffers  (2009)  found  that  the  National  Capital 
Region  Contracting  Center  rated  low  in  contract  process  maturity  but,  found  no 
direct  correlation  between  the  assessed  process  maturity  level  and  the  workforce 
turnover.  She  recommended  that  contracting  processes  be  institutionalize  and 
standardized  (Jeffers,  2009). 

Rendon  has  utilized  the  CMMM  to  assess  a  number  of  different  Army 
contracting  entities  since  2010.  The  assessments  consisted  of  Assessment  of 
Army  Contracting  Command’s  Contract  Management  Processes  (2010) 
evaluating  Aviation  Missile  Command  (AMCOM),  Joint  Munitions  and  Lethality 
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Command  (JM&L)  and  the  National  Capital  Region  (NCR)  and  the  2011 
Assessment  of  Army  Contracting  Command’s  Contract  Management  Processes 
(TACOM  and  RDECOM).  The  results  of  these  studies  determined  that  the 
agencies  surveyed  rated  low  maturity  levels  for  post  award  functions,  specifically 
Contract  Administration  and  Contract  Closeout  (Rendon,  2010;  2011). 

E.  SUMMARY 

In  this  chapter,  we  presented  a  literature  review  on  methods  of  public 
procurement  performance  measurement  and  evaluation.  We  addressed  why 
organizations  should  conduct  measurement,  what  should  be  measured  and  how 
to  measure.  We  also  reviewed  the  Contract  Management  Maturity  Model  that  we 
used  to  conduct  our  study  and  its  previous  applications  with  other  Army 
organizations.  In  the  next  chapter,  we  will  provide  information  on  APG-ACC 
which  was  the  organization  that  participated  in  the  study. 
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III.  ABERDEEN  PROVING  GROUND-ARMY  CONTRACTING 
COMMAND  (ABERDEEN,  MARYLAND) 


A.  INTRODUCTION 

In  this  chapter,  we  provide  a  discussion  on  the  case  setting  for  this 
research.  In  this  chapter  we  will  present  an  overview  of  the  Aberdeen  Proving 
Ground-Army  Contracting  Command.  We  will  discuss  the  organizational 
structure.  We  will  also  discuss  the  types  of  contracts  managed  by  APG-ACC, 
which  includes  service  contracts,  supply  contracts,  and  construction  contracts. 
Finally,  we  will  discuss  organizational  metrics  that  APG-ACC  uses. 

B.  ORGANIZATION 

APG-ACC  is  a  Senior  Executive  Service  (SES)  led  organization  that 
provides  contracting  support  to  a  wide  customer  base  throughout  the  United 
States.  The  organization  is  a  subordinate  unit  of  the  U.S.  Army  Contracting 
Command.  APG-ACC  is  located  in  northern  Maryland  along  the  Chesapeake  Bay 
in  Aberdeen,  Maryland.  The  organization  consist  of  the  Office  of  the  Executive 
Director,  Office  of  the  Deputy  Director,  Chief  of  Staff  G3,  and  16  Divisions  of 
which  13  perform  contracting  duties  (Figure  2).  APG-ACC  is  not  confined  to 
Aberdeen,  MD,  as  there  are  several  contracting  Divisions  within  the  organization 
that  are  in  locations  such  as:  Adelphi,  MD;  Denver,  CO;  Ft.  Belvoir,  VA;  Ft. 
Huachuca,  AZ;  Frederick,  MD;  Natick,  MA;  Orlando,  FL;  Research  Triangle  Park, 
NC;  and  Tobyhanna  Army  Depot,  PA. 

Aberdeen  Proving  Ground-Army  Contracting  Command’s  mission  is  to 
“Provide  responsive,  efficient,  cost-effective  and  compliant  contracts  and 
business  solutions  to  assist  our  customers  with  mission  objectives  in  support  of 
National  Defense”  (www.acc.army.mil,  2014).  The  organization  achieves  its 
mission  by  delivering  contracting  support  to  their  customer  base,  and  garnering 
longstanding  support,  in  the  following  areas:  Research  and  Development; 
Command,  Control,  Communications,  Computer,  Intelligence,  Surveillance  and 
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Reconnaissance  (C4ISR);  Cyber  Security;  Test  and  Evaluation;  Chemical  and 
Biological  Defense;  and  Medical  Research.  In  addition  to  procuring  services  and 
supplies,  the  organization  is  knowledgeable  in  the  areas  of  installation  and  base 
operations,  foreign  military  sales  (FMS),  and  other  transactions. 

Command  and  control  of  the  organization  is  provided  by  the  Director,  who 
is  a  SES,  Department  of  the  Army  (DA)  civilian.  The  current  Director  provides 
leadership  and  guidance  to  over  500  DA  civilians  and  military  Soldiers;  many 
who  are  level  II  or  III  DAWIA  certified.  The  organization  executes  on  average 
more  than  21,000  contracting  actions  in  excess  of  $6.5  billion  per  year;  providing 
seamless  support  to  Program  Executive  Offices  (PEO),  Soldiers,  and  established 
Army  customers. 
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Figure  2.  Aberdeen  Proving  Ground-Army  Contracting  Command 
Organization  Chart.  This  chart  provides  a  visual  representation  of 
how  the  organization  is  structured  from  a  top  down  approach 
(from  APG-ACC,  2014) 
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C.  SERVICES  PROVIDED 

APG-ACC  provides  contacting  support  through  the  purchasing  of  services, 
supplies,  and  construction  from  various  industry,  small  business,  and  commercial 
partners.  APG-ACC’s  primary  customers  are  CECOM,  PEO-C3T,  RDECOM,  and 
Garrison.  As  explained  previously,  APG-ACC’s  three  major  customer 
requirements  are  procurement  of  services,  supplies  and  military  construction.  We 
will  explain  what  each  category  of  procurement  requirements  are  and  an 
example  of  each. 

1.  SERVICE  CONTRACTS 

APG-ACC  like  many  other  contracting  commands  procures  large  amounts 
of  services  for  their  customer  base.  According  to  the  Federal  Acquisition 
Regulation  (FAR)  a  service  contract  is  “a  contract  that  directly  engages  the  time 
and  effort  of  a  contractor  whose  primary  purpose  is  to  perform  an  identifiable  task 
rather  than  to  furnish  an  end  item  of  supply”  (FAR  37.101).  Rendon,  Apte  and 
Apte  pointed  out  that  over  the  last  decade  DOD  has  continued  to  see  an  increase 
in  the  amount  of  services  it  procures  (2012).  For  example,  as  seen  in  Figure  3, 
the  procurement  of  services  in  the  DOD  has  continued  to  increase  in  scope  and 
dollars  in  the  past  decade.  This  is  a  significant  fact  as  the  contract  management 
process  of  these  billions  of  dollars  in  contracts  are  important  to  DOD  and  APG- 
ACC.  The  services  procured  by  APG-ACC  include  installation  level  support, 
facilities  management,  grounds  management,  and  custodial. 
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FIGURE  1.  DoD’S  CONTRACTS  FOR  GOODS  AND  SERVICES 
(2000-2009) 
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Figure  3.  Graphic  representation  of  DOD  dollars  spent  for  goods  and 
services  from  2000  to  2010.  This  graph  shows  the  progression 
of  dollars  spent  for  goods  and  services.  (From  Rendon,  Apte  & 

Apte,  2011,  p.  5). 

2.  SUPPLY  CONTRACTS 

In  addition  to  procuring  services,  APG-ACC  also  procures  large  sums  of 
goods  and  materials.  The  FAR  defines  supplies  as,  “all  property  except  land  or 
interest  in  land.  It  includes  (but  is  not  limited  to)  public  works,  buildings  and 
facilities;  ships,  floating  equipment,  and  vessels  of  every  character,  type,  and 
description,  together  with  parts  and  accessories;  aircraft  and  aircraft  parts, 
accessories,  and  equipment;  machine  tools;  and  the  alteration  and  installation  of 
the  foregoing”  (FAR  2.101).  As  the  contracting  office  for  PEO-C3T,  JPEO 
Chemical  Biological,  and  other  PEOs  on  the  Aberdeen  Proving  Ground 
installation,  APG-ACC  purchases  major  weapon  systems  and  enablers  for  the 
Warfighter.  As  an  example,  during  Fiscal  Year  (FY)  2013,  APG-ACC  awarded 
Harris  Corp.,  Rochester,  N.Y.,  a  $140,700,000  firm-fixed-price,  non-option- 
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eligible,  non-multi-year  contract  for  the  production  of  the  Mid-Tier  Networking 
Vehicular  Radio  (MNVR)  (U.S.  Department  of  Defense,  2014). 

3.  CONSTRUCTION  CONTRACTS 

Although  construction  projects  are  normally  handled  by  the  Army  Corps  of 
Engineers,  Army  Contracting  Commands  such  as  APG-ACC  also  procure 
construction  and  demolition  services  from  industry.  Normally  the  magnitude  of 
construction  will  most  likely  dictate  which  agency  will  procure  the  facilities,  i.e., 
design,  design-bid-build,  and  design-build.  Army  Contracting  Commands  will 
normally  purchase  minor  construction  services.  In  the  management  of  these 
contracts,  APG-ACC  uses  performance  management  metrics  which  will  be 
discussed  next. 

D.  CURRENT  PERFORMANCE  MANAGEMENT  METRICS 

APG-ACC  has  personnel  within  the  Office  of  the  Principle  Assistant 
Responsible  for  Contract  (PARC)  Division,  whose  duties  include  tracking 
organizational  performance  metrics  and  strategic  initiatives.  APG-ACC  collects 
and  monitors  over  70  organizational  performance  metrics  comprised  of  the 
inputs,  processes,  outputs  and  outcomes  model,  previously  discussed  in  Chapter 
II,  on  a  quarterly  basis  categorized  into  the  following:  Human  Capital;  Training 
and  Certification;  Operations  and  Budget,  Pre-Award,  Competition  and 
Socioeconomic;  Post-Award  and  Compliance.  Refer  to  Appendix  B  for  a 
summary  of  APG-ACC’s  metrics.  The  metrics  from  the  Pre-Award  and  Post- 
Award  categories  were  re-categorized  to  align  with  the  contract  management 
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phases  utilized  with  the  CMMM  shown  in  Table  1. 


Procurment  Planning 

Solicitation  Planning 

Solicitation 

Source  Selection 

Contract  Administration 

Contract  Closeout 

Basic 

Structured 

Basic 

Structured 

Basic 

Basic 

#  Peer  Reviews 

#  Peer  Reviews 

#  Peer  Reviews 

#  Peer  Reviews 

#  Peer  Reviews 

#  Peer  Reviews 

#  of  Justification  and 
Approvals 

#  of  Justification  and 
Approvals 

#  of  Justification  and 
Approvals 

Re\arse  Auctions 

Reverse  Auctions 

Reverse  Auctions 

Formal  Source  Selections 
in  Progress 

Protest  Management 

Protest  Management 

CPARS 

CPARS 

UCAs 

#  UCAs 

Contract  Deficiency  Reports 

Contract  Deficiency  Reports 

COR  Appointment 

COR  Appointment 

COR  Appointment 

Overage  Contracts 

Table  1 .  APG-ACC  Performance  Metrics  cross-referenced  with  the 

contract  phases 

E.  SUMMARY 

This  chapter  provided  an  overview  of  APG-ACC’s  organizational  make-up, 
to  include  the  organizations  leadership  and  geographic  locale.  Additionally,  we 
covered  the  contracting  services  APG-ACC  provides  its  customers,  and  current 
contract  management  metrics  used  by  the  organization.  The  next  chapter  will 
consist  of  the  CMMM  assessment  results  and  process  improvement 
recommendations. 
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IV.  ASSESSMENT  RESULTS  AND  PROCESS  IMPROVEMENT 

RECOMMENDATIONS 


A.  INTRODUCTION 

This  chapter  provides  the  results  of  the  Contract  Management  Maturity 
Model  (CMMM)  assessment  conducted  at  Aberdeen  Proving  Ground-Army 
Contracting  Command  (APG-ACC).  We  also  provide  an  in-depth  analysis  of  each 
contract  management  process  area  in  relation  to  the  organization.  Using  the 
analysis  gained  from  the  surveys,  we  will  assign  a  maturity  level  to  determine  the 
command’s  contract  management  process  maturity  level.  We  will  provide  our 
analysis  of  APG-ACC  compliance  reports  and  discuss  any  relationship  to  the 
CMMM  assessment  findings.  Furthermore,  in  this  chapter  we  will  provide 
recommendations  to  APG-ACC  to  help  improve  the  contracting  process 
capability. 

B.  SELECTION  OF  STUDY  PARTICIPANTS 

The  group  targeted  for  this  research  was  Defense  Acquisition  Workforce 
Improvement  Act  (DAWIA)  level  II  and  level  III  certified  1102s  and  military 
equivalents  who  were  directly  involved  in  the  procurement  process  i.e. , 
procurement  analysts  and  contract  specialists.  Using  the  CMMM  as  a  tool  to 
measure  contract  management  process  capability  for  the  organization  provides  a 
benchmark  of  process  maturity  for  the  organization,  of  which  the  workforce  can 
build  upon  to  create  a  more  capable  contract  management  process  capability. 

We  strategically  targeted  DAWIA  level  II  and  level  III  contracting  personnel 
to  provide  a  clearer  picture  of  the  organizations  contract  management  process 
capability,  as  this  group  tends  to  be  the  brain  trust  for  government  contracting. 
DAWIA  level  II  and  III  personnel  have  demonstrated  through  course  completion 
and  experience  that  they  possess  the  requisite  contracting  competency  needed 
to  successfully  manage  contracts  from  Procurement  Planning  to  Contract 
Closeout. 
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c. 


ADMINISTRATION  OF  CMMM  SURVEY 


The  CMMM  survey  was  deployed  August  2014  electronically  and 
remained  open  for  48  days.  The  respondents  were  asked  to  voluntarily  answer  a 
62  question  survey  using  a  Likert  Scale  to  ascertain  the  contract  management 
process  capability  of  APG-ACC.  At  the  end  of  the  survey  period,  we  analyzed 
responses  and  calculated  the  maturity  level  for  each  contract  management 
process  area. 

D.  SURVEY  RESULTS 

There  were  534  eligible  participants  to  take  the  survey;  however  we 
received  responses  from  91  respondents.  This  gave  us  a  response  rate  of 
17.04%.  Table  2  reflects  demographic  data  such  as,  warranted  and  non- 
warranted,  years  of  experience,  and  DAWIA  certification  level.  Additionally,  the 
survey  responses  for  the  contract  management  process  areas  will  be  presented 
along  with  the  corresponding  results  for  the  organization  contract  management 
assessment. 


Table  2.  APG-ACC  RESPONSE  DEMOGRAPHICS 


Total  Eligible 

Completed 

Response 

Warranted 

DAWIA 

Level  1 

DAWIA 
Level  II 

DAWIA 

Level  II 

Yrs  of  Exp 

534 

91 

39 

6 

25 

60 

s  3  =  12 

4  to  8  =  38 

14  to  18=  3 

4  19=  31 

Tables  3,  4,  and  5  provide  the  mean  scores  and  standard  deviations  from 
the  survey  responses  for  each  of  the  contract  management  process  areas.  Also 
reflected  in  the  tables  are  descriptors  for  each  survey  item  as  related  to  the 
contract  management  process  enablers  discussed  in  Chapter  II. 
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Table  3.  APG-ACC  Survey  Item  Responses  for  Procurement 
Planning  and  Solicitation  Planning 


ABERDEEN  PROVING  GROUND  -  ARMY  CONTRACTING  COMMAND 


Key  Process/Item  Number/ 

Key  Process/Item  Number/ 

Description 

Description 

Procurment  Planning 

Mean 

Std  Dev 

a 

Solicitation  Planning 

Mean 

1.1  Process  Strength 

4.09 

1.32 

2.1  Process  Strength 

4.07 

1.2  Process  Strength 

3.58 

1.62 

2.2  Process  Strength 

3.67 

1.3  Process  Strength 

3.41 

1.54 

2.3  Process  Strength 

3.93 

1.4  Process  Results 

3.53 

1.48 

2.4  Process  Results 

4.08 

1.5  Management  Support 

3.82 

1.40 

2.5  Management  Support 

3.86 

1.6  Process  Integration 

3.84 

1.40 

2.6  Process  Integration 

3.88 

1.7  Process  Integration 

3.6 

1.36 

2.7  Process  Integration 

3.71 

1.8  Process  Integration 

3.76 

1.39 

2.8  Management  Support 

3.76 

1.9  Process  Measurement 

3.09 

1.66 

2.9  Process  Measurement 

3.24 

1.10  Process  Measurement 

3.32 

1.49 

2.10  Process  Measurement 

3.52 

Mean  Total 

36.04 

37.72 

Std  Dev 
o 

1.22 

1.51 

1.23 

1.15 

1.26 

1.29 

1.31 

1.22 

1.56 

1.40 
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Table  4.  APG-ACC  Survey  Item  Responses  for  Solicitation  and 

Source  Selection 


ABERDEEN  PROVING  GROUND 

-  ARMY  CONTRACTING  COMMAND 

Key  Process/Item  Number/ 

Key  Process/Item  Number/ 

Description 

Description 

Solicitation 

Mean 

Std  Dev 

0 

Source  Selection 

Mean 

Std  Dev 

0 

3.1  Process  Strength 

3.72 

1.48 

4.1  Process  Strength 

4.16 

1.22 

3.2  Process  Strength 

3.43 

1.66 

4.2  Process  Strength 

3.82 

1.43 

3.3  Process  Strength 

3.46 

1.6 

4.3  Process  Strength 

3.85 

1.28 

3.4  Process  Results 

3.77 

1.16 

4.4  Process  Results 

4.15 

1.01 

3.5  Management  Support 

3.7 

1.35 

4.5  Management  Support 

3.87 

1.24 

3.6  Process  integration 

3.8 

1.25 

4.6  Process  Results 

3.85 

1.32 

3.7  Process  integration 

3.48 

1.47 

4  7  Process  Results 

4.18 

1.02 

3.8  Process  integration 

3.62 

1.31 

4.8  Process  Integration 

3.99 

1.28 

3.9  Process  Measurement 

3.12 

1.57 

4.9  Process  Integration 

3.74 

1.44 

3.10  Process  Measurement 

3.46 

1.45 

4.10  Process  Measurement 

3.3 

1.65 

Mean  Total 

35.56 

4.11  Process  Measurement 

3.5 

1.48 

Mean  Total 

42.41 
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Table  5.  APG-ACC  Survey  Item  Responses  for  Contract 
Administration  and  Contract  Closeout 


ABERDEEN  PROVING  GROUND 

■ARMY  CONTRACTING  COMMAND 

Key  Process/Item  Number/ 

Key  Process/Item  Number/ 

Description 

Description 

Contract  Administration 

Mean 

Std  Dev 

0 

Contract  doseout 

Mean 

Std  Dev 

0 

5.1  Process  Strength 

3.64 

1.35 

6.1  Process  Strength 

3.6 

1.55 

5.2  Process  Strength 

3.35 

1.54 

6.2  Process  Strength 

3.29 

1.73 

5.3  Process  Strength 

3.3 

1.5 

6.3  Process  Strength 

3.55 

1.53 

5.4  Process  Results 

3.6 

1.29 

6.4  Process  Results 

3.99 

1.43 

5.5  Management  Support 

3.38 

1.53 

6.5  Management  Support 

3.01 

1.73 

5.6  Process  Integration 

3.55 

1.52 

6.6  Process  Integration 

3.24 

1.63 

5.7  Process  Integration 

3.43 

1.45 

6.7  Process  Integration 

3.11 

1.67 

5.8  Process  Integration 

3.1 

1.54 

6.8  Process  Measurement 

2.96 

1.77 

5.9  Process  Integration 

2.8 

1.84 

6.9  Process  Measurement 

2.98 

1.76 

5.10  Process  Measurement 

2.99 

1.63 

6.10  Process  Measurement 

2.45 

1.94 

5.11  Process  Measurement 

3.25 

1.52 

Mean  Total 

32.18 

Mean  Total 

36.39 

1.  APG-ACC  Contract  Management  Process  Maturity 


Figure  4  reflects  the  APG-ACC  contract  management  process  maturity 
levels.  As  can  be  seen  in  Figure  4,  the  majority  of  contract  management 
processes  are  at  the  Basic  maturity  level;  while  the  remaining  processes  are  at 
the  Structured  level.  The  following  will  provide  more  in-depth  discussion  of  these 
CMMM  assessment  results. 
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CONTRACT  MANAGEMENT  MATURITY  MODEL© 


MATURITY 

LEVEL 

PROCUREMENT 

PLANNING 

SOLICITATION 

PLANNING 

SOLICITATION 

SOURCE 

SELECTION 

CONTRACT 

ADMIN 

CONTRACT 

CLOSEOUT 

5 

OPTIMIZED 

4 

INTEGRATED 

3 

STRUCTURED 

• 

• 

2 

BASIC 

• 

1 

AD  HOC 

Army  APG  n  =  80 


Figure  4.  Contract  Management  Maturity  Model  Assessment  Results  for 

APG-ACC 

a.  Procurement  Planning,  Solicitation,  Contract  Administration 
and  Contract  Closeout 

APG-ACC’s  Procurement  Planning,  Solicitation,  Contract  Administration 
and  Contract  Closeout  process  areas  were  assessed  at  the  Basic  process 
maturity  level.  This  means  for  these  process  areas  and  related  activities  i.e. , 
conducting  market  research,  amending  solicitation  documents  as  required, 
verifying  contractor  compliance,  and  ensuring  contract  completion 
documentation,  is  at  a  level  where  contract  processes  are  rudimentary  and 
standards  are  established  but  only  required  on  selected  contracts  that  meet 
certain  criteria.  Some  documentation  has  been  set  for  processes  however,  these 
developments  are  not  considered  institutionalized.  Finally,  organizational  policy 


34 


and  standards  established  are  not  required  to  be  consistently  applied  for  any 
contracts  that  have  not  met  certain  criteria  (Rendon,  2011). 

b.  Solicitation  Planning  and  Source  Selection 

The  organization’s  Solicitation  Planning  and  Source  Selection  process 
areas  were  assessed  at  the  Structured  process  maturity  level.  This  means  for 
these  process  areas  and  related  activities  i.e.,  determining  the  procurement 
method,  evaluation  strategy,  and  negotiating  contract  terms  and  conditions, 
these  processes  are  institutionalized  and  formal  documentation  is  established  for 
processes  and  standards.  Documentation  is  present  for  formal  standards  and 
possibly  automated.  There  is  flexibility  in  application  of  processes  and 
documents  to  fit  various  types  of  contract  situations  since  contract  process  use  is 
mandated.  With  this  maturity  level,  the  organizational  leadership  is  well  versed  in 
providing  guidance  and  direction  for  strategy  development  and  issues  related  to 
contract  management  (Rendon,  2011). 

2.  APG-ACC  Process  Enablers 

As  previously  discussed,  each  CMMM  survey  item  is  associated  with  a 
specific  process  enabler  (Process  Strength,  Process  Results,  Management 
Support,  Process  Integration,  and  Process  Measurement).  Using  the  CMMM 
assessment  results  we  analyze  APG-ACC  process  capability  in  terms  of  these 
process  enablers.  Using  the  data  in  Figures  5-9,  an  analysis  of  survey  item 
means  can  be  interpreted  to  reflect  the  implementation  of  those  process 
enablers. 

3.  Process  Strengths 

Figure  5  is  a  graphic  representation  of  the  organization’s  Process 
Strengths.  Of  note,  questions  1.1,  2.1,  and  4.1  mean  scores  were  significantly 
higher  than  the  remaining  questions  for  Process  Strength.  This  analysis  suggests 
the  organization’s  Process  Strength  enablers  are  strongest  in  Procurement 
Planning,  Solicitation  Planning,  and  Source  Selection.  Additionally,  the  analysis 
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suggests  that  APG-ACC  Process  Strength  enablers  are  the  weakest  in  Contract 
Administration  and  Contract  Closeout. 


4.  Process  Results 

Figure  6  represents  the  organization’s  Process  Results.  Questions  2.4, 
4.4,  and  4.7  mean  scores  were  higher  than  the  remaining  questions  posed  for 
Process  Results.  This  analysis  suggests  the  organization’s  Process  Results 
enablers  are  strongest  in  Solicitation  Planning  and  Source  Selection. 
Additionally,  the  analysis  suggests  that  APG-ACC  Process  Results  enablers  are 
the  weakest  in  Procurement  Planning,  and  Contract  Administration. 


APG-ACC  Process  Results 


Figure  6.  APG-ACC  Process  Results 
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5.  Management  Support 

Figure  7  is  a  representation  of  the  organization’s  Management  Support  in 
the  six  areas  of  contract  management  processes.  Question  4.5  mean  score  was 
significantly  higher  than  the  remaining  questions  for  Management  Support.  This 
analysis  suggests  the  organization’s  Management  Support  enablers  are 
strongest  in  Source  Selection.  Additionally,  the  analysis  suggests  that  APG-ACC 
Management  Support  enablers  are  the  weakest  in  Contract  Administration  and 
Contract  Closeout. 


APG-ACC  Management  Support 


Figure  7.  APG-ACC  Management  Support 


6.  Process  Integration 

Figure  8  is  a  representation  of  the  organization’s  Process  Integration. 
Question  4.8  was  significantly  higher  than  the  remaining  questions  for  Process 
Integration.  This  analysis  suggests  the  organization’s  Process  Integration 
enablers  are  strongest  in  Source  Selection.  Additionally,  the  analysis  suggests 
that  APG-ACC  Process  Integration  enablers  are  the  weakest  in  Contract 
Administration  and  Contract  Closeout. 
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APG-ACC  Process  Integration 


Corresponding  Question 


Figure  8.  APG-ACC  Process  Integration 

7.  Process  Measurement 

Figure  9  is  a  representation  of  the  organization’s  Process  Measurement. 
Questions  2.10,  3.10,  and  4.11  were  significantly  higher  than  the  remaining 
questions  for  Process  Measurement.  This  analysis  suggests  the  organization’s 
Process  Measurement  enablers  are  strongest  in  Solicitation  Planning, 
Solicitation,  and  Source  Selection.  Additionally,  the  analysis  suggests  that  APG- 
ACC  Process  Measurement  enablers  are  the  weakest  in  Contract  Administration 
and  Contract  Closeout. 


APG-ACC  Process  Measurement 


Corresponding  Question 


Figure  9.  APG-ACC  Process  Measurement 
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E. 


COMPLIANCE  ANALYSIS 


In  our  literature  review,  we  discussed  other  methods  to  evaluate 
organizational  performance  and  management  processes.  These  methods  include 
evaluating  compliance  to  regulations  and  policy.  We  collected  audit  report 
information  from  the  DOD  Inspector  General  (IG)  for  APG-ACC  to  determine  if 
there  were  any  consistencies  with  the  CMMM  findings.  We  noted  that  in  a  report 
for  2014,  the  DOD  IG  found  that  APG-ACC’s  sub-centers  could  improve  contract 
administration  practices  and  oversight,  specifically  citing  that  contract  officials 
lacked  or  failed  to  have  “adequate  documentation  to  support  incremental  funding; 
properly  authorize  contract  oversight  or  update  quality  assurance  surveillance 
plans;  or  evaluate  contractor  performance  with  standards  that  were  compliant 
with  the  Army  Federal  Acquisition  Supplement”  (DOD  IG,  2014).  This  IG  finding 
seems  consistent  with  the  CMMM  assessment  findings.  The  Basic  maturity  level 
and  the  enabler  of  Process  Strength  was  APG-ACC’s  lowest  mean  score. 

In  a  2013  report,  the  DOD  IG  reviewed  30  sole  source/  non-competitive 
contracts  from  APG-ACC  and  found  28  of  the  contracts  to  be  adequately 
supported.  APG-ACC  was  reported  to  have  conducted  and  documented 
appropriately  market  research  that  was  completed  as  well  as  properly 
documented  when  market  research  was  not  done.  They  also  complied  with 
Justification  and  Approval  requirements  and  applied  the  sole  source  policy 
properly  and  consistently  (DOD  IG,  2013).  These  findings  appear  consistent  with 
our  maturity  rating  of  “Structured”  for  APG-ACC  and  a  high  mean  score  in 
solicitation  documentation. 

The  DOD’s  IG  published  a  consolidated  report  covering  all  the  audits 
conducted  for  acquisition  management  from  Fiscal  Year  (FY)  2003  to  2008 
(2009).  The  issues  were  categorized  into  the  following  areas:  Acquisition  and 
Contract  Award  Decisions,  Contract  types  and  Pricing  and  Contract 
Administration  and  Funding.  APG-ACCs  CMMM  results  appeared  to  be 
inconsistent  with  the  IG  DOD-wide  findings  with  regard  to  the  various  issues 

reported.  Of  note,  Solicitation  Planning  and  Source  Selection  activities  were 
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areas  cited  by  the  IG  as  DOD  wide  issues  that  APG-ACC’s  ratings  of  Structured 
maturity  contradict.  We  might  conclude  that  APG-ACC  and  other  organizations 
performing  well  in  the  IG’s  issue  areas,  adopt  the  sharing  of  best  practices  with 
the  rest  of  the  DOD. 

In  a  2012  GAO  report,  it  was  cited  that  there  is  a  DOD-wide  issue  with 
regard  to  over-aged  contract  closeout.  The  report  found  a  gross  mismatch  in  data 
being  provided  by  Army  Headquarters,  its’  commands  and  contracting  centers 
with  regard  to  the  number  of  over-aged  contracts  requiring  closeout.  It  was 
further  stated  that  inadequate  implementation  plans  linked  to  performance 
metrics  that  compared  “actual  performance  against  planned  results”  was 
contributing  to  DOD’s  inability  to  reduce  the  backlog  (GAO,  2012).  This  seems  to 
be  reflected  in  our  assessment  results  because  not  only  was  Contract  Closeout 
assessed  at  the  Basic  maturity  level,  but  also  Process  Strength,  Management 
Support  and  Process  Measurement  found  to  be  the  lowest  process  enabler 
means  for  Contract  Closeout. 

F.  RECOMMENDATIONS  FOR  CONTRACT  MANAGEMENT  PROCESS 

IMPROVEMENTS  AT  APG-ACC 

This  section  will  focus  on  recommendations  to  the  organization  to  improve 
the  contract  management  processes  at  APG-ACC.  The  recommendations  are 
based  on  the  contract  management  maturity  model  assessment  results  and  what 
improvements  that  can  be  made  to  increase  the  maturity  levels  in  each  of  the  six 
process  areas. 

1.  Procurement  Planning 

Procurement  Planning  was  assessed  at  the  Basic  maturity  level.  In 
addition,  the  process  enabler  of  Process  Results  was  relatively  lower  than  other 
process  enablers.  We  provide  the  following  recommendations  for  improving 
APG-ACC’s  process  maturity  to  the  Structured  level.  For  Procurement  Planning 
process  activities  such  as  conducting  risk  analysis,  conducting  outsource 
analysis,  developing  preliminary  budgets  and  cost  estimates  should  be  fully 
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established  within  the  organization  and  APG-ACC  should  ensure  processes  are 
institutionalized,  and  mandated  throughout  the  entire  organization.  Finally,  senior 
organizational  management  should  be  involved  in  providing  guidance,  direction, 
and  even  approval  of  key  contracting  strategy,  and  decision  related  to  contract 
terms  and  conditions  (Rendon,  2011). 

2.  Solicitation  Planning 

Solicitation  Planning  was  assessed  at  the  Structured  maturity  level. 
Solicitation  Planning  mean  scores  were  consistent  in  all  process  enabler 
categories.  We  provide  the  following  recommendations  for  improving  APG-ACC’s 
process  maturity  to  the  Integrated  level.  For  Solicitation  Planning  process 
activities  such  as  determining  the  procurement  method,  determining  the  contract 
type,  developing  the  solicitation  document,  and  structuring  contract  terms  and 
conditions,  APG-ACC  should  ensure  management  processes  are  fully  integrated 
with  other  organizational  core  processes  such  as  financial  management, 
schedule  management,  and  performance  management.  The  customer  is  also 
integrated  into  the  purchasing  team  and  leadership  utilizes  performance  metrics 
to  evaluate  the  contract  management  process  and  make  decisions  (Rendon, 
2011). 


3.  Solicitation 

Solicitation  was  assessed  at  the  Basic  maturity  level.  In  addition,  the 
process  enablers  of  Process  Strength,  Process  Results,  and  Process  Integration 
were  relatively  lower  than  other  process  enablers.  We  provide  the  following 
recommendations  for  improving  APG-ACC’s  process  maturity  to  the  Structured 
level.  For  Solicitation  process  activities  such  as  conducting  advertising  of  the 
procurement  opportunity,  conducting  a  pre-proposal  conference,  and  developing 
and  maintaining  a  qualified  bidder’s  list  should  be  fully  established  within  the 
organization  and  APG-ACC  should  ensure  processes  are  institutionalized,  and 
mandated  throughout  the  entire  organization.  Finally,  senior  organizational 
management  should  be  involved  in  providing  guidance,  direction,  and  even 
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approval  of  key  contracting  strategy,  and  decision  related  to  contract  terms  and 
conditions  (Rendon,  2011). 

4.  Source  Selection 

Source  Selection  was  assessed  at  the  Structured  maturity  level.  Mean 
scores  for  Source  Selection  were  consistent  in  all  process  enabler  categories. 
We  provide  the  following  recommendations  for  improving  APG-ACC’s  process 
maturity  to  the  Integrated  level.  For  Source  Selection  process  activities  such  as 
applying  evaluation  criteria  to  the  management,  cost,  and  technical  proposals, 
negotiating  with  suppliers,  and  executing  the  contract  award  strategies,  APG- 
ACC  should  ensure  management  processes  are  fully  integrated  with  other 
organizational  core  processes  such  as  financial  management,  schedule 
management,  and  performance  management.  The  customer  is  also  integrated 
into  the  purchasing  team  and  leadership  utilizes  performance  metrics  to  evaluate 
the  contract  management  process  and  make  decisions  (Rendon,  2011). 

5.  Contract  Administration 

Contract  Administration  was  assessed  at  the  Basic  maturity  level.  In 
addition,  Contract  Administration  scored  lower  in  all  process  enabler  areas 
except  Process  Results.  We  provide  the  following  recommendations  for 
improving  APG-ACC’s  process  maturity  to  the  Structured  level.  For  Contract 
Administration  process  activities  such  as  conducting  a  pre-performance 
conference,  monitoring  the  contractors  work  results;  managing  the  contract 
change-control  process,  and  measuring  contractor’s  performance  should  be  fully 
established  within  the  organization  and  APG-ACC  should  ensure  processes  are 
institutionalized,  and  mandated  throughout  the  entire  organization.  Finally,  senior 
organizational  management  should  be  involved  in  providing  guidance,  direction, 
and  even  approval  of  key  contracting  strategy,  and  decision  related  to  contract 
terms  and  conditions  (Rendon,  2011). 
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6. 


Contract  Closeout 


Contract  Closeout  was  assessed  at  the  Basic  maturity  level.  In  addition, 
Contract  Closeout  scored  lower  in  all  process  enabler  areas.  We  provide  the 
following  recommendations  for  improving  APG-ACC’s  process  maturity  to  the 
Structured  level.  For  Contract  Closeout  process  activities  such  as  processing  of 
government  property  dispositions,  final  acceptance  of  products  and  services,  and 
documentation  of  the  contractor’s  final  past-performance  report,  should  be  fully 
established  within  the  organization  and  APG-ACC  should  ensure  processes  are 
institutionalized,  and  mandated  throughout  the  entire  organization.  Finally,  senior 
organizational  management  should  be  involved  in  providing  guidance,  direction, 
and  even  approval  of  key  contracting  strategy,  and  decision  related  to  contract 
terms  and  conditions  (Rendon,  2011). 

G.  SUMMARY 

In  this  chapter,  we  provided  the  results  from  the  CMMM  assessment.  We 
determined  the  APG-ACC  contract  management  process  maturity  level  for  each 
of  the  CMMM  process  areas.  We  also  analyzed  APG-ACC  process  enablers  as 
they  relate  to  the  maturity  levels.  We  also  compared  the  CMMM  assessment 
results  with  recent  GAO  and  DOD  IG  compliance  inspections.  Lastly,  we 
provided  APG-ACC  recommendations  for  improving  contract  management 
process  capability.  In  the  next  chapter  we  will  summarize  this  report,  answer  the 
research  questions,  provide  our  conclusions,  and  recommendations  for  further 
research. 
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V.  SUMMARY,  CONCLUSIONS,  AND  AREAS  FOR  FURTHER 

RESEARCH 

A.  INTRODUCTION 

In  this  chapter,  we  summarize  the  background  and  purpose  of  our 
research.  We  then  conclude  our  research  by  presenting  our  findings  to  our 
research  questions.  Finally,  based  on  our  findings,  we  recommend  areas  for 
further  research. 

B.  SUMMARY 

The  Acquisition  workforce  has  undergone  significant  changes  due  to  U.S. 
participation  in  conflicts  across  the  globe.  The  increased  requirement  for 
contracted  services  and  the  shortage  of  federal  contracting  workforce  has 
resulted  in  issues  of  accountability,  transparency  and  value  for  money,  which  are 
important  to  the  public’s  interest.  The  federal  response  to  these  issues  over  the 
years  has  been  to  increase  training  of  the  contracting  workforce  to  improve 
individual  competency  however,  the  issues  still  prevail.  What  is  needed  is  an 
emphasis  on  organizational  contract  management  process  capability.  The 
purpose  of  our  research  was  to  assess  the  contract  management  process 
capability  using  the  CMMM  and  provide  recommendations  to  where  APG-ACC 
can  improve  contract  management  processes. 

C.  CONCLUSION 

1.  What  is  the  Contract  Management  Process  Maturity  Level  for 
APG-ACC  in  Each  of  the  Six  Process  Areas  of  the  CMMM? 

The  results  of  the  CMMM  assessment  indicated  that  APG-ACC  is 
operating  at  the  Basic  maturity  level  for  Procurement  Planning,  Solicitation, 
Contract  Administration  and  Contract  Closeout.  The  organization  is  performing  at 
the  Structure  maturity  level  in  Solicitation  Planning  and  Source  Selection. 
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2.  What  Process  Improvement  Opportunities  Are  Available  for 
APG-ACC,  Based  on  the  CMMM  Assessment  at  APG? 

Using  the  CMMM  assessment  results,  we  made  recommendations  for 
improvements  most  notably  in  the  area  of  process  measurement.  We  identified 
bolstering  the  capture  of  best  practices  and  lessons  learned  as  well  as  increasing 
systematic  measurement  of  efficiency  and  effectiveness  metrics  across  all 
phases  of  contracting  could  improve  overall  contract  management  performance 
within  the  organization. 

3.  What  Metrics  does  APG  Currently  Use  to  Assess  Contract 
Management  Performance? 

APG-ACC  utilizes  a  database  populated  with  the  measurement  model  of 
inputs,  processes,  outputs  and  outcomes.  They  collect  information  quarterly  from 
all  of  their  respective  divisions.  We  further  categorized  their  metrics  to  coincide 
with  the  six  contract  management  phases. 

4.  How  Do  Compliance  Reports  of  APG-ACC  Compare  to  the 
CMMM  Results? 

We  compared  APG-ACC’s  CMMM  assessment  results  to  recent  GAO  and 
DOD  IG  reports  to  assess  the  compliance  element  of  performance  measurement. 
We  found  there  appeared  to  be  consistencies  with  recent  published  findings  from 
GAO  and  DOD  IG  determined  recommendations  for  improvement.  We  also  noted 
APG-ACC  may  be  performing  better  in  relation  to  the  rest  of  DOD  regarding 
some  issues  cited,  suggesting  that  sharing  of  best  practices  and  lessons  learned 
with  other  organizations  could  assist  the  entire  DOD. 

D.  AREAS  FOR  FURTHER  RESEARCH 

Based  on  our  research  findings,  we  recommend  the  following  areas  for 
further  research: 
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Cross-reference  the  contract  files  of  an  the  organization  and  maturity 
model  data  to  see  if  the  processes  areas  match  the  physical  records  to  confirm 
what  the  organization  reported  they  are  doing. 

Conduct  interviews  of  employees  in  addition  to  the  survey  to  provide 
further  input  to  what  the  organization  may  be  doing  to  hinder  or  further  its’ 
progression. 

Conduct  an  assessment  that  examines  demographic  data  of  the  survey 
participants  to  the  results  of  the  CMMM  to  determine  a  correlation  in  experience 
or  DAWIA  level  with  organizational  maturity. 

Compare  the  survey  results  with  other  organizations  in  the  Army  to 
determine  trends  and  knowledge  sharing  opportunities. 

Initiate  a  reassessment  of  the  organization  after  implementation  of 
process  improvement  recommendations. 
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B.  APG-ACC  PERFORMANCE  METRICS 
Human  Capital 

1 .  Percentage  of  1 102  workforce  that  are  interns 

2.  Total  Fill  Rate  of  employees  to  positions 

3.  Percent  of  operating  division  personnel  that  directly  support  the  mission  of 
awarding/  administrating  contracts 

4.  Number  of  personnel  hired  from  outside  of  the  organization 

5.  Number  of  personnel  who  left  the  organization 

6.  Number  of  personnel  who  moved  divisions  within  the  organization  (gains  and 
losses) 

7.  Total  number  of  gains  plus  the  number  of  losses 

8.  Number  in  Intern  Program  Status 

9.  Number  Support  Contractors 

Training  and  Certifications 

1 .  Percentage  of  the  organization  that  is  DAWIA  Certified 

2.  Number  of  DAWIA  Certified  Within  24-month  grace  period  and  non-certified 

3.  Number  of  DAWIA  Certified  Beyond  24/40  months,  non-certified 

4.  Percentage  of  the  acquisition  workforce  meeting  Continuous  Learning  Point 
requirements 

5.  Percentage  of  the  acquisition  workforce  with  current  Individual  Development 
Plans 

6.  Percentage  of  the  acquisition  workforce  on  Performance  Standards 

7.  Number  of  Competitive  Award  Nominations 

8.  Number  of  DAU  No-Shows 

9.  Number  of  DAU  Attritions 


10. Total  DAU  Resident  Course  Attendees 
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1 1 .  Percentage  of  the  acquisition  workforce  that  has  completed  Civilian  Educating 
System  (CES)  Foundations 

12.  Percentage  of  employees  that  have  completed  CES  Basic 

13.  Percentage  of  employees  that  have  completed  CES  Intermediate 

14.  Percentage  of  employees  that  have  completed  CES  Advanced 

15.  Percentage  of  supervisors  that  have  completed  Supervisor  Development 
Course 

Operations  and  Budget 

1.  Cost  to  obligate  a  dollar 

2.  Dollars  per  Person 

3.  Number  of  actions  per  person 

4.  Dollars  Obligated  per  Budget  Dollar 

5.  Operating  Funds  Received  Compared  to  Projected  Funds  Received 

6.  Operating  Funds  Obligated  (To  Funds  Received) 

7.  Operating  Funds  Obligated  (To  Total  Operating  Budget) 

Pre-Award/  Award 

1 .  Number  of  Peer  Reviews  Completed  per  quarter 

2.  Number  of  Justifications  and  Approvals 

3.  Number  of  Bridge  Justifications  and  Approvals 

4.  Percentage  of  Time  and  Material  /  Labor  Hour  Dollars  obligated 

5.  Percentage  of  actions  completed  using  reverse  auctioning  procedures 

6.  Number  of  Formal  Source  Selections  in  Process 
Competition/Socio-Economic 

1 .  Percentage  of  dollars  competed  versus  total  dollars  obligated 

2.  Percentage  of  dollars  awarded  to  Small  Business 
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3.  Percentage  of  dollars  awarded  to  Small  Disadvantaged  Business 

4.  Percentage  of  dollars  awarded  to  Service  Disabled,  Veteran-Owned  Small 
Business 

5.  Percentage  of  dollars  awarded  to  Women-Owned  Small  Business 

6.  Percentage  of  dollars  awarded  to  Hub-zone  Small  Business 

Post  Award 

1 .  Number  of  protests  received 

2.  Number  of  protest  that  required  corrective  action  or  dismissed  due  to 
corrective  action 

3.  Number  of  protest  that  required  corrective  action  or  dismissed  without 
corrective  action 

4.  Number  of  protest  sustained 

5.  Number  of  ratification 

6.  Number  of  Freedom  of  Information  Act  (FOIA)  requests  received 

7.  Number  of  Freedom  of  Information  Act  (FOIA)  requests  completed 

8.  Number  of  Freedom  of  Information  Act  (FOIA)  requests  open 

9.  Percentage  of  Undefinitized  Contract  Actions  (UCAs)  greater  than  180  days 

10.  Percentage  of  Contract  Deficiency  Reports  (CDRs)  Resolved 

1 1 .  Number  of  unresolved  CDRs  over  1 1  days 

12.  Percentage  of  Contracting  Officer  Representative  (COR)  appointments 

13.  Total  number  of  active  contracts  where  the  completion  has  not  passed 

14.  Number  of  contracts  per  person 

15.  Number  of  Contracts  Closed 

16.  Number  of  overaged  contracts 

17.  Number  of  overaged  contracts  awaiting  closeout 

18.  Number  of  new  on-time  audit  in  the  Contact  Audit  Follow  Up  (CAFU)  system 
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19.  Number  of  new  reportable  audits  closed  in  CAFU 

20.  Total  number  of  overage  audits  in  CAFU 

Compliance 

1 .  Percentage  of  allocated  billing  official  accounts  reviewed 

2.  Percentage  of  delinquent  Government  Purchase  Card  (GPC)  payments 

3.  Percentage  of  billing  official  accounts  within  the  proper  span  of  control  ratio 

4.  Percentage  of  service  contracts  coded  as  performance  based  on  the  contract 
action  report  (CAR) 

5.  Percentage  of  dollars  associated  with  actions  missing  a  CAR 

6.  Number  of  actions  missing  a  CAR 
Percentage  of  travel  vouchers  filed  within  5  days 

7.  Percentage  of  actions  awarded  adhering  to  paperless  contracting 

8.  Percentage  of  folders  with  one  paperless  file 
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